Maneuver Control based on Reinforcement Learning for Automated Vehicles in An Interactive Environment

Operating a robot safely and efficiently can be considerably challenging in an interactive and complex environment. Other surrounding agents may be cooperative or adversarial in their interactions with the robot. It will be desirable to develop control strategies that can enable the robot agent to handle diverse situations and respond with appropriate behaviors in an interactive environment. In this paper, we focus on automated vehicles, and propose a reinforcement learning based approach to train the vehicle agent for safe, comfortable, and efficient maneuvers under interactive driving situations. Particularly, we design a form of the Q-function approximator that consists of neural networks but also has a closed-form greedy policy. In this way, we avoid the complication of invoking an additional function that learns to take actions, as in actorcritic algorithms. Additionally, we formulate the vehicle control maneuvers with continuous state and action space to enhance the practicability and feasibility of the proposed approach. We test our algorithm in simulation with a challenging use case, the lane change maneuver. Results show that the vehicle robot successfully learns a desirable driving policy that allows it to drive safely, comfortably, and efficiently in complex driving scenarios.